[TensorRT EP] Refactor OrtTensorRTProviderOptions initialization and make it easy to add new field #17617
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
jywu-msft
reviewed
Sep 22, 2023
jywu-msft
reviewed
Sep 22, 2023
|
i kicked off python packaging pipeline to test this PR |
|
fix python lint checks |
jywu-msft
previously approved these changes
Sep 29, 2023
|
The old tensorrt_provider_factory.h only contained a nAPI OrtSessionOptionsAppendExecutionProvider_Tensorrt declaration and I moved it to onnxruntime_c_api.h
I think most of the applications should already include onnxruntime_c_api.h, so we should be okay.
Will double check inference-examples and kick off nuget-zip packages pipeline as well.
From: George Wu ***@***.***>
Sent: Sunday, October 1, 2023 8:26 AM
To: microsoft/onnxruntime ***@***.***>
Cc: Chi Lo ***@***.***>; Author ***@***.***>
Subject: Re: [microsoft/onnxruntime] [TensorRT EP] Refactor OrtTensorRTProviderOptions initialization and make it easy to add new field (PR #17617)
are there any code examples , inference-examples that need to be updated now that tensorrt_provider_factory.h is removed?
-
Reply to this email directly, view it on GitHub<#17617 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANBP7RANNSUT5ABPB7FOQGDX5GDPJANCNFSM6AAAAAA467CUWY>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
jywu-msft
approved these changes
Oct 5, 2023
chilo-ms
added a commit
that referenced
this pull request
Oct 16, 2023
We removed tensorrt_provider_factory.h in the [PR](#17617). Need to remove the copy of this file when cmake install.
jchen351
pushed a commit
that referenced
this pull request
Oct 18, 2023
We removed tensorrt_provider_factory.h in the [PR](#17617). Need to remove the copy of this file when cmake install.
kleiti
pushed a commit
to kleiti/onnxruntime
that referenced
this pull request
Mar 22, 2024
…make it easy to add new field (microsoft#17617) Two major modifications of this PR: 1. Refactor OrtTensorRTProviderOptions initialization and make it easy to add new field. 2. Make Python API capable of using TensorRT plugins by adding new Python binding api `register_tensorrt_plugins_as_custom_ops`. (It needs to register ep's custom op domain before model load. For C++ API, it's slightly different, when calling SessionOptionsAppendExecutionProvider_TensorRT_XX, it appends cutom op domain to session option. Later ORT can register custom op domain from session option before model loading)
kleiti
pushed a commit
to kleiti/onnxruntime
that referenced
this pull request
Mar 22, 2024
We removed tensorrt_provider_factory.h in the [PR](microsoft#17617). Need to remove the copy of this file when cmake install.
siweic0
pushed a commit
to siweic0/onnxruntime-web
that referenced
this pull request
May 9, 2024
…make it easy to add new field (microsoft#17617) Two major modifications of this PR: 1. Refactor OrtTensorRTProviderOptions initialization and make it easy to add new field. 2. Make Python API capable of using TensorRT plugins by adding new Python binding api `register_tensorrt_plugins_as_custom_ops`. (It needs to register ep's custom op domain before model load. For C++ API, it's slightly different, when calling SessionOptionsAppendExecutionProvider_TensorRT_XX, it appends cutom op domain to session option. Later ORT can register custom op domain from session option before model loading)
|
Is |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Two major modifications of this PR:
register_tensorrt_plugins_as_custom_ops. (It needs to register ep's custom op domain before model load. For C++ API, it's slightly different, when calling SessionOptionsAppendExecutionProvider_TensorRT_XX, it appends cutom op domain to session option. Later ORT can register custom op domain from session option before model loading)